Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle truncated PES packets #37

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

tmm1
Copy link
Contributor

@tmm1 tmm1 commented Jan 21, 2022

Simple fix for #35

Unfortunately this is breaking some tests which are creating and operating on malformed packets.

--- FAIL: TestParsePESData (0.00s)
    --- FAIL: TestParsePESData/with_header (0.00s)
        data_pes_test.go:407:
            	Error Trace:	data_pes_test.go:407
            	Error:      	Not equal:
            	            	expected: &astits.PESData{Data:[]uint8{0x64, 0x61, 0x74, 0x61}, Header:(*astits.PESHeader)(0x1381690)}
            	            	actual  : &astits.PESData{Data:[]uint8{0x64, 0x61, 0x74, 0x61, 0x73, 0x74, 0x75, 0x66, 0x66}, Header:(*astits.PESHeader)(0xc0000534a0)}

            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1,4 +1,4 @@
            	            	 (*astits.PESData)({
            	            	- Data: ([]uint8) (len=4) {
            	            	-  00000000  64 61 74 61                                       |data|
            	            	+ Data: ([]uint8) (len=9) {
            	            	+  00000000  64 61 74 61 73 74 75 66  66                       |datastuff|
            	            	  },
            	            	@@ -62,3 +62,3 @@
            	            	   }),
            	            	-  PacketLength: (uint16) 67,
            	            	+  PacketLength: (uint16) 9,
            	            	   StreamID: (uint8) 1
            	Test:       	TestParsePESData/with_header
--- FAIL: TestParseData (0.00s)
    data_test.go:47:
        	Error Trace:	data_test.go:47
        	Error:      	Not equal:
        	            	expected: []*astits.DemuxerData{(*astits.DemuxerData)(0xc000073770)}
        	            	actual  : []*astits.DemuxerData{(*astits.DemuxerData)(0xc000073720)}

        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -24,4 +24,4 @@
        	            	   PES: (*astits.PESData)({
        	            	-   Data: ([]uint8) (len=4) {
        	            	-    00000000  64 61 74 61                                       |data|
        	            	+   Data: ([]uint8) (len=9) {
        	            	+    00000000  64 61 74 61 73 74 75 66  66                       |datastuff|
        	            	    },
        	            	@@ -85,3 +85,3 @@
        	            	     }),
        	            	-    PacketLength: (uint16) 67,
        	            	+    PacketLength: (uint16) 9,
        	            	     StreamID: (uint8) 1
        	Test:       	TestParseData
FAIL
FAIL	github.com/asticode/go-astits	0.008s
FAIL

Specifically, the with_header PES test advertises a packet length of 67 even though it only has 9 bytes of data:

go-astits/data_pes_test.go

Lines 258 to 265 in 473f66c

"with_header",
func(w *astikit.BitsWriter, withStuffing bool, withCRC bool) {
packetLength := 67
stuffing := []byte("stuff")
if !withStuffing {
packetLength -= len(stuffing)
}

go-astits/data_pes_test.go

Lines 329 to 332 in 473f66c

stuffing := []byte("stuff")
w.Write([]byte("data")) // Data
if withStuffing {
w.Write(stuffing) // Stuffing

I tried to fix these tests but they make a lot of assumptions and reuse fixtures like pesWithHeaderBytes() which creates quite the rabbit hole.

@tmm1
Copy link
Contributor Author

tmm1 commented Jan 21, 2022

Simple fix for #35

My first attempt was more complex: fancybits@a55f10a

That approach also works, but the same tests fail for the same reason.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.03%) to 77.24% when pulling b1ef54e on tmm1:truncated-pes-adjust into 473f66c on asticode:master.

@asticode
Copy link
Owner

Specifically, the with_header PES test advertises a packet length of 67 even though it only has 9 bytes of data

Are you sure this is an error? 🤔 packet length is supposed to indicate the number of bytes after this field, optional header included.

@asticode
Copy link
Owner

Regarding the content of the PR, I don't really like the idea of ignoring the problem and rewriting the packet length.

As you suggested here, I would implement the CorruptedPacketHandler logic and make the corrupted packet go through it.

@eric
Copy link
Contributor

eric commented Nov 15, 2022

Fixed the issue with the tests not passing.

@asticode
Copy link
Owner

Fixed the issue with the tests not passing.

Thanks @eric, but as I mentioned in my previous comment, I'd rather go the CorruptedPacketHandler route in this case 👍

@ayhangenc
Copy link

Hi,

I'd like to share my case which might be related, I set up a muxer from a live stream into srt mpegts via astits.newmuxer and during muxing I have error messages both on go app and ffplay(receiver);
SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 185 bytes of payload: only 182 is available...
on app log and then
panic: runtime error: slice bounds out of range [1319:1316]
goroutine 12 [running]:
bufio.(*Writer).Write(0xc00029e100, {0xc0002801e1?, 0x10014c675?, 0x10014c675?}) /usr/local/go/src/bufio/bufio.go:670 +0x1c8
github.com/asticode/go-astikit.(*BitsWriter).write(0xc0002a42d0, {0xc0002801e1, 0x1, 0x1}) /Users/user/go/pkg/mod/github.com/asticode/[email protected]/binary.go:142 +0x3d
on console...
also on ffplay;
[mpegts @ 0x7fae05714180] Packet corrupt (stream = 0, dts = 6652800).
[h264 @ 0x7fae06a3d280] concealing 2720 DC, 2720 AC, 2720 MV errors in P frame
[aac @ 0x7fae068416c0] channel element 0.5 is not allocated f=0/0

I just wonder how to overcome this? What would be CorruptPacketHandler case for muxer?

Thanks in advance

AyhG

@asticode
Copy link
Owner

@ayhangenc could you share the next 10 or 20 lines of the panic stack?

@ayhangenc
Copy link

ayhangenc commented Jun 13, 2023

Hi @asticode ,

here is the complete dump of the log screen for the app.
it'a a basic rtsp to srt gateway. test source is ffmpeg and the srt consumer is ffplay currently.
I have a lot of error messages on ffplay while video is frozen before panic. Panic is not instant though, it takes a while.
hope this helps..
thnak you.

the error message on ffplay is this:

...
[mpegts @ 0x7fbf19f16c00] changing packet size to 192    0B f=0/0   
[mpegts @ 0x7fbf19f16c00] changing packet size to 188    0B f=0/0   
[mpegts @ 0x7fbf19f16c00] changing packet size to 192    0B f=0/0   
[mpegts @ 0x7fbf19f16c00] changing packet size to 188    0B f=0/0   
[mpegts @ 0x7fbf19f16c00] changing packet size to 204    0B f=0/0   
[mpegts @ 0x7fbf19f16c00] changing packet size to 188    0B f=0/0   
1906.04 A-V:520.879 fd= 123 aq=    0KB vq=    0KB sq=    0B f=0/0   
...

and the app log:

% go run .
2023/06/13 10:39:55 loading config files from config folder ./config!...
2023/06/13 10:39:55 Running RTSP Client!... live2
2023/06/13 10:39:55 Running RTSP Client!... live1
2023/06/13 10:39:55 Socket  connected!... &{129186975 false 0xc000236090 127.0.0.1 5808 map[blocking:0 latency:250 mode:caller payloadsize:1316 streamid:CAM1 transtype:live] 2 1456 -1}
2023/06/13 10:39:55 Socket  connected!... &{129186974 false 0xc0002b0090 127.0.0.1 5809 map[blocking:0 latency:250 mode:caller payloadsize:1316 streamid:CAM2 transtype:live] 2 1456 -1}
2023/06/13 10:39:55 IDR Not Present Error: 
2023/06/13 10:39:55 Encoding video access units into muxer error: IDR Not Present Error... 
2023/06/13 10:39:55 IDR Not Present Error: 
2023/06/13 10:39:55 Encoding video access units into muxer error: IDR Not Present Error...
2023/06/13 10:39:55 IDR Not Present Error: 
2023/06/13 10:39:55 Encoding video access units into muxer error: IDR Not Present Error... 
...
2023/06/13 10:40:09 IDR Not Present Error: 
2023/06/13 10:40:09 Encoding video access units into muxer error: IDR Not Present Error...  
2023/06/13 10:40:09 IDR Not Present Error: 
2023/06/13 10:40:09 Encoding video access units into muxer error: IDR Not Present Error...  
2023/06/13 10:40:16 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 272 bytes of payload: only 182 is available...  
2023/06/13 10:40:16 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:40:16 Encoding video access units into muxer error: SRT Send Error... 
2023/06/13 10:40:16 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 293 bytes of payload: only 182 is available...  
2023/06/13 10:40:16 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:40:16 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:40:44 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 333 bytes of payload: only 182 is available...  
2023/06/13 10:40:44 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:40:44 Encoding audio access units into muxer error: SRT Send Error... 
2023/06/13 10:42:04 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 356 bytes of payload: only 184 is available...  
2023/06/13 10:42:04 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:42:04 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:42:14 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 187 bytes of payload: only 182 is available...  
2023/06/13 10:42:14 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:42:14 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:42:14 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 187 bytes of payload: only 182 is available...  
2023/06/13 10:42:14 SRT Audio ErrorChannel active... 1001
 
2023/06/13 10:42:14 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:43:08 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 183 bytes of payload: only 182 is available...  
2023/06/13 10:43:08 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:43:08 Encoding video access units into muxer error: SRT Send Error... 
2023/06/13 10:43:19 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 193 bytes of payload: only 182 is available...  
2023/06/13 10:43:19 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:43:19 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:43:45 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 143 bytes of payload: only 140 is available...  
2023/06/13 10:43:45 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:43:45 Encoding video access units into muxer error: SRT Send Error... 
2023/06/13 10:43:55 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 236 bytes of payload: only 184 is available...  
2023/06/13 10:43:55 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:43:55 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 350 bytes of payload: only 182 is available...  
2023/06/13 10:43:55 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:43:55 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:43:55 Encoding video access units into muxer error: SRT Send Error... 
2023/06/13 10:44:17 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 321 bytes of payload: only 184 is available...  
2023/06/13 10:44:17 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:44:17 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:44:34 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 310 bytes of payload: only 182 is available...  
2023/06/13 10:44:34 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:44:34 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:44:34 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 305 bytes of payload: only 182 is available...  
2023/06/13 10:44:34 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:44:34 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:44:39 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 197 bytes of payload: only 182 is available...  
2023/06/13 10:44:39 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:44:39 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:44:39 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 55 bytes of payload: only 30 is available...  
2023/06/13 10:44:39 SRT Audio ErrorChannel active... 1001
 
2023/06/13 10:44:39 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:44:39 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 258 bytes of payload: only 182 is available...  
2023/06/13 10:44:39 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:44:39 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:45:03 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 229 bytes of payload: only 184 is available...  
2023/06/13 10:45:03 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:45:03 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 343 bytes of payload: only 182 is available...  
2023/06/13 10:45:03 Encoding audio access units into muxer error: SRT Send Error...   
2023/06/13 10:45:03 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:45:03 Encoding video access units into muxer error: SRT Send Error... 
2023/06/13 10:45:13 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 245 bytes of payload: only 184 is available...  
2023/06/13 10:45:13 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:45:13 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:46:08 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 219 bytes of payload: only 184 is available...  
2023/06/13 10:46:08 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:46:08 Encoding video access units into muxer error: SRT Send Error... 
2023/06/13 10:46:23 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 213 bytes of payload: only 182 is available...  
2023/06/13 10:46:23 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:46:23 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:46:55 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 222 bytes of payload: only 182 is available...  
2023/06/13 10:46:55 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:46:55 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:46:55 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 142 bytes of payload: only 68 is available...  
2023/06/13 10:46:55 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:46:55 Encoding video access units into muxer error: SRT Send Error...  
2023/06/13 10:47:48 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 185 bytes of payload: only 184 is available...  
2023/06/13 10:47:48 SRT Audio ErrorChannel active... 1001
 
2023/06/13 10:47:48 Encoding audio access units into muxer error: SRT Send Error...  
2023/06/13 10:48:44 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 156 bytes of payload: only 153 is available...  
2023/06/13 10:48:44 SRT Video ErrorChannel active...  127.0.0.1
2023/06/13 10:48:44 Encoding video access units into muxer error: SRT Send Error... 
2023/06/13 10:48:44 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 175 bytes of payload: only 22 is available...  
2023/06/13 10:48:44 SRT Audio ErrorChannel active... 1002
 
2023/06/13 10:48:44 Encoding audio access units into muxer error: SRT Send Error...  
panic: runtime error: slice bounds out of range [120:0]

goroutine 52 [running]:
bytes.(*Buffer).Write(0x3?, {0xc000280318?, 0xc00004a970?, 0x10039f53b?})
        /usr/local/go/src/bytes/buffer.go:172 +0xd6
github.com/asticode/go-astikit.(*BitsWriter).write(0xc000298320, {0xc000280318, 0x1, 0x1})
        /Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:142 +0x3d
github.com/asticode/go-astikit.(*BitsWriter).flushBsCache(...)
        /Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:169
github.com/asticode/go-astikit.(*BitsWriter).writeFullByte(0x0?, 0x20?)
        /Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:179 +0x7e
github.com/asticode/go-astikit.(*BitsWriter).writeByteSlice(0xc000298320, {0xc000599180, 0xa8, 0xc00004aaa0?})
        /Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:130 +0x89
github.com/asticode/go-astikit.(*BitsWriter).Write(0xc000298320?, {0x1003e5cc0?, 0xc00004aa50?})
        /Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:75 +0xa5
github.com/asticode/go-astits.writePESData(0x100405d00?, 0xc000282570?, {0xc000599180?, 0x1aa?, 0x1aa?}, 0x8?, 0xb6?)
        /Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/data_pes.go:465 +0xb3
github.com/asticode/go-astits.(*Muxer).WriteData(0xc0002846e0, 0xc00004acb8)
        /Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/muxer.go:230 +0x4dd
X.7/gateway.(*SRT).Encode(0xc0000e2580, {0xc0003c9aa0, 0x4, 0x10038d28a?}, {0x0?, 0x10005687f?, 0x10004e552?}, 0x7db35b0a00)
        /Users/anyuser/GolandProjects/X.7/gateway/srt.go:175 +0x10cd
X.7/gateway.(*Gateway).RunRTSP.func1(0xc000292140?)
        /Users/anyuser/GolandProjects/X.7/gateway/rtsp.go:127 +0x191
github.com/bluenviron/gortsplib/v3.(*clientFormat).readRTPUDP(0xc0002983c0, 0xc000282870?)
        /Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_format.go:117 +0x185
github.com/bluenviron/gortsplib/v3.(*clientMedia).readRTPUDPPlay(0xc000296480, {0xc000176000, 0xc00004af20?, 0x5c1})
        /Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_media.go:291 +0xae
github.com/bluenviron/gortsplib/v3.(*clientUDPListener).runReader(0xc0002b80e0)
        /Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_udp_listener.go:195 +0x1e5
created by github.com/bluenviron/gortsplib/v3.(*clientUDPListener).start
        /Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_udp_listener.go:153 +0xaa
exit status 2

@asticode
Copy link
Owner

Can you open an issue including test data which would allow us to reproduce the problem so that it could be traced more easily?

@asticode asticode force-pushed the master branch 2 times, most recently from 106bbe5 to 0df190a Compare July 27, 2023 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants